Skip to content

Feat/preview missing status#1103

Merged
frankrousseau merged 2 commits into
cgwire:mainfrom
nledez:feat/preview-missing-status
Jun 9, 2026
Merged

Feat/preview missing status#1103
frankrousseau merged 2 commits into
cgwire:mainfrom
nledez:feat/preview-missing-status

Conversation

@nledez

@nledez nledez commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

Problem

  • renormalize-movie-preview-files failures left PreviewFile.status stuck on processing, so Kitsu kept showing affected previews as processing forever even though the
    renormalize pass had given up.
  • The single broken status conflated two very different cases: a movie that exists but cannot be normalized (encode/ffmpeg error, transient storage hiccup) versus a movie
    whose source binary is simply gone from object storage. Operators could not target the irrecoverable rows without retrying the recoverable ones too.
  • sync-full-files re-downloaded every preview every time, including rows known to be unusable, wasting bandwidth and time on previews that will never play.

Solution

  • renormalize-movie-preview-files now marks failed rows on exit: source-binary-missing → new missing status, other failures → broken. The 3-preview reproducer that prompted
    this no longer stays as processing.
  • New internal status missing added to PreviewFile.STATUSES. No DB migration needed — ChoiceType is enforced at ORM level only.
  • EXPOSE_MISSING config flag (env var, default false) gates the API exposure of the new status. With the default, missing is folded back to broken in serialize(), present()
    and present_minimal() so existing API/Kitsu clients see no change. This flag is intentionally temporary: it gives us time to (1) announce the new status publicly, (2) ship the
    matching Kitsu UI/UX changes, and (3) let third-party clients adopt. Once Kitsu speaks missing natively, EXPOSE_MISSING will be removed and missing will be surfaced
    unconditionally.
  • renormalize-movie-preview-files gains a --all-missing flag (sibling of --all-broken / --all-processing); --all-broken stays broken-only, --all-missing is the new operator
    escape hatch to retry rows whose source has since been restored.
  • sync-full-files gains --skip-broken and --skip-missing flags (default off, so behavior is unchanged) to let operators exclude unusable rows from large pulls.
  • New service helper preview_files_service.set_preview_file_as_missing(); internal queries that filter ("broken", "processing") now include "missing" so it is treated the same
    as broken for "needs attention" purposes.
  • Tests added for the missing→broken serialization remap (both EXPOSE_MISSING values) and for the --all-missing filter behavior.

Comment thread zou/app/config.py Outdated
nledez and others added 2 commits June 9, 2026 12:18
Add a new `missing` PreviewFile status that means "the source movie is
gone from storage, so renormalization is no longer possible". Until
now, both irrecoverable rows and genuine encode failures were lumped
together as `broken`, leaving operators no way to retry only the
recoverable ones and forcing them to keep hitting Swift 404s on every
renormalize pass.

The new status is persisted via SQLAlchemy ChoiceType (no DB CHECK
constraint, no migration), and exposed verbatim through the API and
serialization helpers so the wire format gains a new value.

`zou renormalize-movie-preview-files`:
- Mark renormalization failures so rows do not stay in `processing`
  forever: source-binary-missing now sets `missing`, every other
  failure keeps setting `broken`.
- Add `--all-missing` next to `--all-broken` and `--all-processing`,
  so each status can be targeted independently. `--all-broken` no
  longer touches `missing` rows.

`zou sync-full-files`:
- Add `--skip-broken` and `--skip-missing` so operators can exclude
  unusable previews from large pulls; defaults preserve previous
  behavior of syncing every row.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@nledez nledez force-pushed the feat/preview-missing-status branch from f51007d to 872825e Compare June 9, 2026 12:17
@frankrousseau frankrousseau merged commit 4edb120 into cgwire:main Jun 9, 2026
14 checks passed
@nledez nledez deleted the feat/preview-missing-status branch June 9, 2026 12:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants